Skip to main content

Client-side load balancing

what is?

Client-side load balancing is a feature that allows gRPC clients to distribute load optimally across available servers.

Client-side load balancing involves the client being aware of multiple endpoints and choosing a different endpoint for each gRPC call. It is a suitable option when low latency is critical because there is no intermediary between the client and the service, allowing direct communication. However, one potential drawback is that the client must maintain a list of available endpoints.

In contrast, lookaside client load balancing involves storing the load balancing state in a centralized location. Clients regularly request information from this location to make informed load balancing decisions.

Configuring client side load balancing

Client-side load balancing is configured when a channel is created. There are two components to consider when using load balancing:

  1. The resolver, which resolves the addresses for the channel. Resolvers support getting addresses from an external source. This is also known as service discovery.
  2. The load balancer, which creates connections and picks the address that a gRPC call will use. Built-in implementations of resolvers and load balancers are included in Grpc.Net.Client. Load balancing can also be extended by writing custom resolvers and load balancers.

Addresses, connections and other load balancing state is stored in a GrpcChannel instance. In order to have work correctly - the channel must be reused when making gRPC calls.

Resolvers​

A resolver is configured using the address a channel is created with. The URI scheme of the address specifies the resolver. A channel doesn't directly call a URI that matches a resolver. Instead, a matching resolver is created and used to resolve the addresses. There are 2 resolver types:

  1. dns This corresponds to the DnsResolverFactory type. Resolves addresses by querying the hostname for DNS address records.
    A records

    This correspond to the A records. It returns a 32-bit IPv4 address, most commonly used to map hostnames to an IP address of the host, but it is also used for DNSBLs, storing subnet masks in RFC 1101, etc.

  2. static This corresponds to the DnsResolverFactory type. Resolves addresses by querying the hostname for DNS address records.

Static resolvers​

StaticResolverFactory

  • Doesn't call an external source. Instead, the client app configures the addresses.
  • Is designed for situations where an app already knows the addresses it calls.
Title
var factory = new StaticResolverFactory(addr => new[]
{
    new BalancerAddress("localhost", 80),
    new BalancerAddress("localhost", 81)
});

title

Some load balancing configuration uses dependency injection (DI). Apps that don't use DI can create a ServiceCollection instance.

If an app already has DI setup, like an ASP.NET Core website, then types should be registered with the existing DI instance. GrpcChannelOptions.ServiceProvider is configured by getting an IServiceProvider from DI.

var services = new ServiceCollection();
services.AddSingleton<ResolverFactory>(factory);

var channel = GrpcChannel.ForAddress(
    "static:///my-example-host",
    new GrpcChannelOptions
    {
        Credentials = ChannelCredentials.Insecure,
        ServiceProvider = services.BuildServiceProvider()
    });
var client = new Greet.GreeterClient(channel);

Configuring the load balancer

Once we decided what is the resolver we want to use, we need to specify what is the load balancing policy that we need. We can specify a policy by using a service config.

Title
var channel = GrpcChannel.ForAddress(
    "dns:///my-example-host",
    new GrpcChannelOptions
    {
        Credentials = ChannelCredentials.Insecure,
        ServiceConfig = new ServiceConfig {
        LoadBalancingConfigs = {
                                    new RoundRobinConfig()
                                 }
         }
    });

Service config gRPC

A service config in gRPC is a type that allows service owners to publish parameters to be automatically used by all clients of their service.

  • Round-Robin​

    Attempts to connect to all addresses. gRPC calls are distributed across all successful connections using round-robin logic. It corresponds to a RoundRobinLoadBalancerFactory type

  • Pick first​

    Attempts to connect to addresses until a connection is successfully made. gRPC calls are all made to the first successful connection. It corresponds to a PickFirstLoadBalancerFactory type